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Abstract.  A  means  of  measuring  machine  intelligence  is  presented.  The  technique  is 
based  on  geometric  procedures  and  works  best  on  relative  comparisons  across  different 
entities,  rather  than  absolute  comparisons  of  intelligence. 
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I.  INTRODUCTION 

Defining,  evaluating,  and  obtaining  viable 
metrics  for  the  measurement  of  autonomy, 
machine  intelligence  quotient  (MIQ),  or 
intelligence,  in  general,  is  a  nontrivial  task  [1- 
9] .  It  is  generally  agreed  that  intelligence 
must  be  a  high  dimensional  vector  involving 
multiple  attributes  of  a  human  or  machine 
(Meystel  [1]).  Defining  the  relevant 
dimensions  is  also  not  a  trivial  task  and  much 
controversy  exists.  Even  the  discussion  on 
how  testing  on  intelligence  is  performed  with 
humans  creates  controversy  on  which  mental 
abilities  constitute  intelligence.  The  relevant 
issues  include  whether  the  IQ  obtained,  e.g. 
by  the  Stanford-Binet  Intelligence  Scale  or 
the  Wechsler  Scales,  are  fair  measures. 
Additional  controversy  also  exists  that  certain 
less  privileged  racial,  ethnic,  or  social  groups 
do  not  have  fair  representations  on  the  test 
questions  pertinent  to  their  living 
environments. 

Albus  [2]  defines  intelligence  as  having 
many  dimensions.  He  also  recognizes  degrees 
or  levels  of  intelligence.  Some  of  the 
influencing  parameters  in  describing  features 
of  intelligence  for  unmanned  ground  vehicles 
include,  but  are  not  limited  to: 

(1)  The  computational  power  and  memory 
capacity  of  the  system’s  brain  (or  computer), 

(2)  The  sophistication  of  the  processes 
the  system  employs  for  sensory  processing, 
world  modeling,  behavior  generation,  value 
judgment,  communication,  and, 


(3)  The  quality  and  quantity  of 
information  and  values  the  system  has 
stored  in  its  memory. 

The  measure  of  intelligence  is  also  predicated 
on  the  success  in  solving  problems, 
anticipating  the  future,  and  acting  so  as  to 
maximize  the  likelihood  of  achieving  goals. 
Obviously  intelligence  is  goal  oriented  and 
related  to  success.  The  presumption  is  that 
different  levels  of  intelligence  produce 
dissimilar  probabilities  of  success  in  the 
accomplishment  of  specific  missions. 

In  studying  autonomous  systems  [3],  there 
are  numerous  (analogous)  systems  that  can  be 
examined  for  attributes  both  within  and  across 
processes  that  relate  to  autonomy.  Some  of 
these  systems  include  living  things  (birds, 
fish,  insects),  intelligent  highway  vehicle 
systems,  mobile  robots,  control  of  satellites 
in  orbit,  underwater  vehicles  systems, 
helicopters,  tanks,  human-machine  interfaces, 
unmanned  air  vehicles,  swarms  of  robots,  and 
a  host  of  other  processes.  In  studying 
unmanned  air  vehicle  systems  (UAVs)  [8,10] 
autonomy  is  desired  since  the  goal  is  to 
maximize  the  ratio  of  UA Vs/operators  for  a 
number  of  important  reasons.  The  advantages 
include  the  significant  reduction  in  cost,  the 
elimination  of  the  need  to  include  a  life 
support  system  (significantly  reducing  fuel 
and  weight  requirements),  decreased 
vulnerability  if  the  UAV  is  shot  down  or 
captured,  enhancing  reliability  and  robustness 
with  multiple  opportunities  to  achieve  a 
mission,  as  well  as  other  important  traits. 
Again,  in  the  design  of  UAVs,  it  is  desired  to 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

SEP  2001 


2.  REPORT  TYPE 


3.  DATES  COVERED 

00-00-2001  to  00-00-2001 


4.  TITLE  AND  SUBTITLE 

An  Autonomous  Metric  (Polytope-Convex  Hull)  For  Relative 
Comparisons  of  MIQ 

6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Research  Laboratory ,AFRL/HECP, Wright  Patterson 
AFB, OH, 45433 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 


11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

Proceedings  of  the  2001  Performance  Metrics  for  Intelligent  Systems  Workshop  (PerMIS  ?01),  Mexico 
City,  Mexico  on  4  Sep  2001 


14.  ABSTRACT 

see  report 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 

OF  PAGES 

19a.  NAME  OF 

RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

7 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


have  a  metric  to  compare  within  and  across 
different  systems  on  the  level  of  autonomy  or 
intelligence  designed  in  the  aircraft. 

It  was  pointed  out  in  [4]  that,  at  best,  a 
measure  of  machine  intelligence  (MIQ)  is  a 
relative  metric  and  it  is  difficult  to  have  an 
absolute  measure.  This  paper  will  discuss  a 
relative  means  of  determining  how  to 
contrast  across  different  machines  for 
comparative  intelligence  or  autonomy.  The 
goal  is  to  have  an  objective  measure  to 
demonstrate  that  one  machine  has  higher  or 
lower  degrees  of  intelligence  or  autonomy  in 
comparison  to  another  machine.  Thus  the 
designer  can  rate  different  machines  in  terms 
of  their  relative  MIQ  and  investigate  trade¬ 
offs  between  gain  in  MIQ  versus  cost  and  the 
benefits  derived.  It  is  cautioned  that  MIQ  is 
very  mission  specific,  and  unless  the  mission 
can  be  accomplished  with  the  appropriate 
level  of  success,  then  the  machine  may  still 
not  be  appropriate.  In  other  words,  the 
appropriate  tool  has  to  be  able  to  perform  the 
given  task.  Success  in  a  mission  is  the  final 
measure  that  demonstrates  that  a  machine  has 
the  appropriate  MIQ  for  a  given  application. 

To  understand  the  metric  introduced  here, 
some  basics  need  to  be  reviewed  and 
discussed  to  better  grasp  how  the  measure  of 
MIQ  was  constructed  herein. 

II.  Some  Basic  Definitions 

To  understand  the  ensuing  definition  of 
MIQ,  some  basic  concepts  need  to  be 
reviewed.  We  present  the  fundamental 
nomenclature  via  key  definitions. 

Definition  1  -  Convexity: 

A  subset  A  of  R11  is  convex  if,  for  any  vectors 
x  and  y  in  A  and  scalars  r  and  s  with  r  >  0  and 
s  >  0,  r  +  s  =  1 ,  then  every  point  r  x  +  s  y 
remains  in  A.  In  other  words,  if  we  have  a 
convex  set  (2  dimensions)  A  with  two  points  x 
and  y,  then  if  we  draw  a  line  from  the  point  x 
to  y,  every  point  on  the  line  remains  inside  the 


Figure  1  -  The  Convex  Set  A  of  a  Circle 


surface  A.  Figure  1  illustrates  a  circle  in 
which  the  points  x  and  y  lie  inside  the  circle. 
Drawing  a  hne  from  the  point  x  to  the  point  y 
still  remains  inside  the  circle  A.  Also,  every 
point  along  the  line  joining  x  to  y  also  lies 
within  the  set  A  and  no  point  on  the  line  is 
outside  the  set  A.  Other  examples  of  convex 
spaces  in  3  dimensions  include  a  cube,  a 
sphere,  etc.  A  cube  is  defined  as  follows: 

"V 


Cube  =  A  =  { 


:  lx,l<l,  lx,  I<1,  lx3l<l}  (1) 


x 


3 


It  is  also  worthwhile  to  look  at  a  surface 
which  is  not  convex.  Example  1  describes  a 
set  of  points,  which  is  not  convex. 

Example  1-A  set  of  points  in  a  nonconvex  set 

The  set  A  of  points  in  R2  defined  by: 


A  =  { 


:  X[  >  0,x2  >03  (X|3  +x23  )2  <  1}  (2) 


Figure  2  is  a  plot  of  the  nonconvex  surface  A. 
It  is  easily  seen  that  a  line  cannot  be  drawn 
between  any  two  points  x  and  y  in  A  and 
have  every  point  on  the  line  joining  the  points 
still  lie  in  A.  Thus  the  surface  A  in  figure  2  is 
a  nonconvex  surface.  Sometimes  it  is 
necessary  to  prove  a  surface  is  convex  by  the 
definition  of  its  constituent  elements.  The 


following  alternative  definition  is  useful  for 
this  purpose. 


Figure  2  -  A  Nonconvex  Set  A 


Alternative  Definition  of  Convexity: 

A  function  f(x)  is  convex  if  for  all  x,  y  and  X 
such  that:  0  <  X  <  1, 

f&  x  +  (1-  X)y)  <X  f(x)  +  (1  -  X)  f(y)  (3) 

The  next  three  definitions  will  prepare  for  the 
appropriate  definition  of  MIQ.  Definition  2 
refers  to  the  outer  surface  (Convex  Hull)  that 
encloses  the  convex  set. 

Definition  2  -  Convex  Hull: 

The  convex  cover  (Convex  Hull)  of  a 
convex  set  is  what  bounds  the  outside  of  the 
convex  set.  For  figure  1,  it  is  the 
circumference  of  the  circle.  For  the  cube  of 
equation  (1),  the  Convex  Hull  is  the  six 
surfaces  of  the  cube.  To  define  the  Convex 
Hull  more  formally: 

Let  B  be  any  subset  of  Rn  and  CH(B)  is  the 
convex  hull  of  B  if  it  contains  all  the  convex 
combinations  of  the  elements  of  B ,  i.e. 

CH(5)  =  {  x  :  there  are  elements  xi,  X2,  ..., 
xn  in  B  such  that  x  is  a  convex  combination  of 
all  of  the  Xj  elements  considered} 

Hence  the  Convex  Hull  is  the  outside 
bounding  surface  of  the  convex  set.  The  next 
definition  generalizes  this  concept  to  multiple 
dimensions.  Polytopes  have  many  definitions, 


e.g.  with  respect  to  classes  of  polynomials 
[11],  with  respect  to  matrices  [12],  and  also 
with  reference  to  general  convex-compact  sets 
[13].  Here  the  choice  is  made  to  use  the  term 
poly  tope  with  respect  to  geometric  figures. 

For  a  set  of  points  in  Rn  where  n  >  2,  the 
concept  of  convexity  is  now  extended  to 
multiple  dimensions. 

Definition  3  -  Poly  tope: 

Given  the  subset  A  of  Rn  which  is  a 
polytope  if,  for  any  vectors  x  and  y  in  A  and 
scalars  r  and  s  with  r  >  0  and  s  >  0,  r  +  s  =  1, 
then  every  point  r  x  +  s  y  still  remains  in  A. 
This  generalizes  for  n  >2  and  all  points  can 
be  connected  in  A.  Figure  3  illustrates  a 
triangle  as  a  2  dimensional  convex  set  and 
figure  4  generalizes  this  result  to  3 
dimensions.  The  goal  is  to  increase  n  to  any 
number  greater  than  2  and  triangles  or 
geometric  figures  with  vertices  will  be  used  in 
each  dimension. 


Figure  3  -  A  Triangle  as  a  2 -dimensional  Polytope 


Goals  Achieved 


Uncertainty  in  the  Environment 


Figure  4  -  A  Triangle  Extended  as  a  3  -dimensional  Polytope 


Definition  4  -  MIQ  as  a  Poly  tope: 

The  prior  definitions  have  provided  some 
valuable  tools  to  help  in  the  definition  of  a 
measure  of  MIQ  in  multiaxes,  as  is  necessary 
since  intelligence  is  such  a  multidimensional 
process.  There  is  a  3  step  process  in 
developing  this  methodology. 

Step  1:  Consider  a  minimum  of  3  attributes 
for  a  2  dimensional  definition  of  MIQ. 

Step  2:  Generalize  this  result  to  4  or  more 
attributes  in  this  2-dimensional  (planar  space). 
In  the  two  dimensional  space,  the  map  now 
extrapolates  with  any  number  of  features 
necessary  to  complete  the  mission. 

Step  3:  The  last  step  takes  the  generalization 
to  a  third  or  higher  dimension.  In  all  cases  all 
the  figures  constructed  are  Convex  Hulls  or 
polytopes.  Thus  comparisons  can  always  be 
made  within  any  dimension  involving  two  or 
more  machines  to  be  considered.  To  explain 
this  better,  figure  5  illustrates  the  Step  1 
process  with  the  3  attributes  of  intelligence 
[2]  being  defined  as:  goals  achieved  (task 
performance),  uncertainty  in  the  environment, 
and  sensors  available.  Figure  6  now 


Figure  5  -  First  Definition  of  MIQ  with  3  Axes  (Intelligence  Attributes) 

extrapolates  the  previous  figure  to  include  a 
total  of  5  attributes  in  the  planar  dimension 
with  the  addition  of  two  more  attributes  of 
intelligence  selected  including:  actuators 
controlled  and  a  priori  knowledge.  Finally 
figure  7  generalizes  to  3  dimensions  with  the 
addition  of  three  additional  intelligence 
attributes  in  the  third  dimension,  including: 
accuracy  level,  time  efficiency,  and  energy 


Goals  Achieved 


Figure  6  -  Generalization  of  Figure  5  to  now  Include  5  Attributes 


Goals  Achieved 


Uncertainty  in  the  Environment 


Figure  7  -  Generalization  of  Figure  6  to  3  Dimensions  With  8  Attributes. 

efficiency  [9].  To  this  point,  the  process  has 
been  an  abstraction;  in  the  next  section  a 
comparison  is  made  of  relative  examples  to 
illustrate  how  to  use  this  methodology. 

Methodology  to  Compare  Across  Machines 

To  illustrate  how  to  use  the  methodology, 
four  examples  are  considered  with  (presumed) 
increasing  levels  of  intelligence  (machine  or 
nonmachine).  They  include: 

(1)  A  toaster. 

(2)  A  washing  machine  with  fuzzy  logic 
to  detect  quality  of  cleaning. 

(3)  An  insect  (ant). 

(4)  A  human  operator. 

Due  to  the  complexity  of  representation, 
figure  8  portrays  a  comparison  of  the  washing 
machine  with  fuzzy  logic  to  the  toaster  using 
the  simplified  planar  representation 


Uncertainty  in  the  Environment 


Figure  8-  Comparison  of  a  Toaster  and  Clothes  Washer  With  Fuzzy  Logic 


Figure  9-  Comparison  of  a  Human,  Ant  and  Toaster 

introduced  in  figure  5.  Obviously  the  more 
intelligent  machine  is  further  displaced  from 
the  origin  and  due  to  the  convexity  of  the 
polytope,  it  is  seen  that,  in  general,  the  fuzzy 
logic  system  appears  to  have  greater  machine 
intelligence  (area  measure).  In  figure  9,  the 
evaluation  of  MIQ  is  now  made  between  the 
mixture  of  living  things  and  machines.  The 
comparison  involves  a  human,  an  ant,  and  a 
toaster.  Here  the  relative  hierarchy  is 
specified  by  the  amount  of  area  or  volume 
contained  in  each  polytope.  Thus  the 
intelligence  measure  is  very  relative  (not 
absolute)  to  compare  across  living  things  and 
machines.  To  summarize  the  results  so  far,  the 
following  paradigm  is  suggested  on  how  to 
synthesize  this  MIQ  metric: 

Steps  in  Synthesizing  the  MIQ  Paradigm: 

(i)  For  the  specific  mission,  define  the  axes  of 
the  polytope  to  be  relevant  to  the  performance 
of  the  mission  under  consideration  (e.g.  a 
toaster  cannot  clean  a  rug,  nor  can  a  washing 
machine  toast  a  piece  of  bread). 

(ii)  Define  the  scales  of  each  axes  of  the 
polytope  relevant  to  the  mission  of  interest. 

(iii)  Plot  alternative  machines  on  the  same 
axes. 

(iv)  The  hypervolume  resulting  will  provide  a 
relative  (not  absolute)  comparison  of  the 
efficacy  of  a  particular  machine  to  perform 
certain  missions. 

Recall  there  is  no  absolute  standard 
(however,  an  existing  machine  could  be  a 


baseline  for  comparison  purposes)  and,  at 
best,  the  relevance  of  each  machine  to 
perform  a  specific  mission  can  be  better 
understood  via  this  procedure. 

III.  Summary  and  Conclusions 

Using  properties  such  as  convexity  and 
relative  measures  of  machine  intelligence,  the 
effectiveness  to  perform  specific  missions 
under  various  conditions  can  be  determined. 

It  is  difficult  to  obtain  an  absolute  measure  of 
MIQ  but  by  comparison  to  baseline  or 
existing  machines  in  use,  there  is  some  value 
in  the  relative  comparison.  The  results  can  be 
extended  to  any  level  of  complexity  by 
considering  convex  polytopes  in  a  multiple 
dimensional  space. 
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